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Abstract. An obstacle _ff C M", n > 3, n odd, is called trapping if there 
exists at least one generalized bicharacteristic 7(t) of the wave equation stay- 
ing in a neighborhood of K for all t > 0. We examine the singularities of the 
scattering kernel s{t,8,Lij) defined as the Fourier transform of the scattering 
amplitude a(\,9,Lu) related to the Dirichlet problem for the wave equation in 
O = R" \ K. We prove that if K is trapping and ■y(t) is non-degenerate, then 
there exist reflecting (oJm , Sm)-rays Sm, m 6 N, with sojourn times Tm — » +oo 
as m — > oo, so that — S sing supp s(t,8m,'jJm), Vm G N. We apply this 
property to study the behavior of the scattering amplitude in C 



1. Introduction 

Let K C {x E M", \x\ < p}, n > 3, n odd, be a bounded domain with C°° 
boundary dK and connected complement fl = \ K. Such K is called an obstacle 
in M". In this paper we consider the Dirichlet problem for the wave equation 
however in a similar way one can deal with other boundar value problems. Given 
two directions {9,uj) G §"^^ x consider the outgoing solution Vs{x,X) of the 

problem 

Ua + \'^)vs ^0 in 

\ws +e-'^<^'"> = ondK, 

satisfying the so called (iA) - outgoing Sommcrfcld radiation condition: 

v,{re,X) = (a{X,e,Lo) + o(^^^y x^re, as|a;|=r^oo. 

The leading term a(A,(?,w) is called scattering amplitude and we have the following 
representation 

aiX,e,co) ^ ^^^ly^Zl |^^(iA(K:.),g)e'^<--^-> -e'^<--^>^(x, A))dg. , (1.1) 

where (•, •) denotes the inner product in M" and 1^(0;) is the unit normal to a; 6 dK 
pointing into fl (see [S], [T5]). 
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Throughout this note we assume that 9 ^ oj. The scattering kernel s{t,9,uj) is 
defined as the Fourier transform of the scattering amphtude 

/ / iA \ ("-i)/2 \ 

s{t,9,to)^Tx^t[[^) aiX,9,co)y 

where (^J^x^t^^^ (t) = (27r)-i / e'*^ip{X)dX for functions (p e S(R). Let V{t, x; lo) be 
the solution of the problem 

(92 _ A)V = in K X 17, 
= onRxdK, 

V\t<-p^S{t~{x,Lu)). 

Then we have 

s{a, 9, uj) = (-l)(«+i)/22-"7ri-" / d'^-'^d^V{{x, 9) - a, x; Lo)dS^ , 

JdK 

where the integral is interpreted in the sense of distributions. 

The singularities of s{t,9,u!) with respect to t can be observed since at these 
times we have some non negligible picks of the scattering amplitude. For example, 
if K is strictly convex, for fixed 9 ^ lo we have only one singularity at t = — T-y 
related to the sojourn time of the unique (a;, 6')-reflecting ray 7 (see [8]). For gen- 
eral non-convex obstacles the geometric situation is much more complicated since 
we have different type of rays incoming with direction uj and outgoing in direction 
9 for which an asymptotic solution related to the rays is impossible to construct. 
In many problems, such as those concerning local decay of energy, behavior of the 
cut-off resolvent of the Laplacian, the existence of resonances etc. the difference 
between non-trapping and trapping obstacles is quite significant. In recent years 
many authors studied mainly trapping obstacles with some very special geometry 
and the case of several strictly convex disjoint obstacles has been investigated both 
from mathematical and numerical analysis point of view. 

In this work our purpose is the study the obstacles having at least one (w, 9)- 
trapping ray 7 which in general could be non-reflecting (see Section 2 for the defi- 
nition of an {uj,9)-iay). No assumptions are made on the geometry of the obstacle 
outside some small neighborhood of 7 and no information is required about other 
possible {ll>, 0)-rays. Our aim is to examine if the existence of 7 may create an 
infinite number of delta type singularities — > 00 of s{—t, 9m, t^m), in contrast 
to the non-trapping case where s{t,9,oj) is C°° smooth for \t\ > Tq > and all 
{9,uj) € X On the other hand, it is important to stress that the scat- 

tering amplitude and the scattering kernel are global objects and their behavior 
depends on all (Li;,^)-rays so any type of cancellation of singularities may occur. 
The existence of a trapping ray influences the singularities of s(t, 9, w) if we assume 
that 7 is non-degenerate which is a local condition (see Section 3). Thus our result 
says that from the scattering data related to the singularities of s{t,9,uj) we can 
"hear" whether K is trapping or not. 

The proof of our main result is based on several previous works [13] , [M] , [15] , 
[16j , [19j , and our purpose here is to show how the results of these works imply the 
existence of an infinite number of singularities. The reader may consult |18j for a 
survey on the results mentioned above. 
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2. Scattering kernel 

We start with the definition of the so called reflecting {uj, 0)-rays. Given two 
directions (w, 9) G S"^^ x S"~^, consider a curve 7 £ having the form 

j = UT=oh, m>l, 

where li = [x^, x^+i] are finite segments for i = l,...,m — 1, x^G dK, and Iq (resp. 
Im) is the infinite segment starting at xi (resp. at Xm) and having direction —uj 
(resp. 9). The curve 7 is called a reflecting {uj, 0)-ray in Q if for i = 0, 1, m — 1 
the segments h and Z^+i satisfy the law of reflection at x^+i with respect to dK. 
The points xi, Xm are called reflection points of 7 and this ray is called ordinary 
reflecting if 7 has no segments tangent to dK. 

Next, we deflne two notions related to (cj, 0)-rays. Fix an arbitrary open ball 
Uo with radius a > containing K and for ^ G S"^^ introduce the hyperplane 
orthogonal to ^, tangent to Ua and such that ^ is pointing into the interior of the 
open half space with boundary containing Ua. Let tt^ : R" — > be the 
orthogonal projection. For a reflecting (w, 0)-ray 7 in with successive reflecting 
points a;i, the sojourn time of 7 is defined by 

m — 1 

T'7 = ||7rt^(a;i) - + ^ - a;i+i|| + \\xra - Tr-e{xra)\\ - 2a. 
1=1 

Obviously, T-y + 2a coincides with the length of the part of 7 that lies in H^^ n H-e. 
The sojourn time T-y does not depend on the choice of the ball Uq and 

Ty = {xi.Uj) + ^ \\xi - Xi+i\\ - {Xm,9) . 
i=l 

Given an ordinary reflecting (cj,0)-ray 7 set — 7r^(xi). Then there exists 
a small neighborhood Wj of u.y in such that for every u G there is an 
unique direction 9{u) G S"""'^ and points xi{u), ...,a;m(u) which are the successive 
reflection points of a reflecting {u,9{u))-ia.y in Q with n^{xi{u)) — u (see Figure 
1). We obtain a smooth map 

J^:Wj3u — > 9{u) G S"-i 

and dJ-y{u-y) is called a differential cross section related to 7. We say that 7 is 
non-degenerate if 

det dJry{Uy) ^ . 

The notion of sojourn time as well as that of differential cross section are well known 
in the physical literature and the deflnitions given above are due to Guillcmin [5]. 

For non-convex obstacles there exist {to, 0)-rays with some tangent and/or gliding 
segments. To give a precise deflnition one has to involve the generalized bicharac- 
teristics of the operator D = df — deflned as the trajectories of the generalized 
Hamilton flow Tt in generated by the symbol J27=i & ~ ''''^ of □ (see [11] for a 
precise deflnition). In general, J-'t is not smooth and in some cases there may exist 
two different integral curves issued from the same point in the phase space (see 
P5] for an example). To avoid this situation in the following we assume that the 
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Figure 1. 



following generic condition is satisfied. 

(Q) If for (a;,^) £ T*{dK) the normal curvature of dK vanishes of infinite 
order in direction ^, then dK is convex at x in direction f . 

Given cr — {x,£,) G T*{n) \ {0} ~ T*{il), there exists a unique generalized 
bicharacteristic {x{t),£,{t)) G r*(ri) such that a;(0) — x, S,{Q) — ^ and we define 
Tt{x,C) = {x{t),^{t)) foralU G R(see 11 ). We obtain a flow : t*{n) — > f*{fl) 
which is called the generalized geodesic flow on T*{Q). It is clear, that this flow 
leaves the cosphere bundle S*(fl) invariant. The flow J-t is discontinuous at points 
of transversal reflection at Tgj^{Q) and to make it continuous, consider the quotient 
space T*(ri)/ ~ of T*{n) with respect to the following equivalence relation: p cr if 
and only if p = cr or p, cr G Tgj^^fl) and either limjyo -^t (p) ~ f or limtx^o -^t (p) = o"- 
Let Efe be the image of S*{n) in T*{U)/ ~. The set Ef, is called the compressed 
characteristic set. Melrose and Sjostrand (111]) proved that the natural projection 
of on T*(ri)/ ^ is continuous. 

Now a curve 7 = {x{t) G : i G K} is called an {uj,9)-ray if there exist real 
numbers ti < t2 so that 



^{t)^{x{t),mes*{n) 
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is a generalized bicharacteristic of □ and 

^(t) = cj for i < ti, ^(t) = 6* for t > t2, 

provided that the time t increases when we move afong 7. Denote by C(^i^£^(fl) the 
set of all [ijj,9)-rays in fJ. The sojourn time Ts of (5 G C(^^ g-j(fl) is defined as the 
length of the part of 6 lying in n H^g. 

It was proved in [T^], [3] (cf. also Chapter 8 in [TT and TU]) that for a; 7^ 6' we 

have 

sing suppt sit,e,uj) C {-T^ : 7 G (2.1) 

This relation was established for convex obstacles by Majda [9\ and for some Rie- 
mann surfaces by Guillemin [S]. The proof in [T^], [3] deals with general obstacles 
and is based on the results in [111 concerning propagation of singularities. 

In analogy with the well-known Poisson relation for the Laplacian on Riemann- 
ian manifolds, l|2.ip is called the Poisson relation for the scattering kernel, while 
the set of all Tj, where 7 G C(^^_g-)(fl), {oj, 9) G S^^^ x S""-'^, is called the scattering 
length spectrum of K. 

To examine the behavior of s{t, 9,uj) near singularities, assume that 7 is a fixed 
non-degenerate ordinary reflecting (u;,6')-ray such that 

/ Ts for every S G q^.e)^ \ {7}- (2.2) 

By using the continuity of the generalized Hamiltonian flow, it is easy to show that 

(-T^ - e, -T^ + e) n sing supp^ s(t, 9, uj) = {-T^} (2.3) 

for e > sufficiently small. For strictly convex obstacles and uj 9 every {uj, d)-ia,y 
is non-degenerate and (|2.3p is obviously satisfied. For general non-convex obstacles 
one needs to establish some global properties of {uj, 0)-rays and choose {uj, 9) so that 
p.3p holds. The singularity of s{t,9,uj) at t = —T^ can be investigated by using 
a global construction of an asymptotic solution as a Fourier integral operator (see 
[B], [Uj and Chapter 9 in [15 )i and we have the following 

Theorem 2.1. f|12|') Lei 7 be a non- degenerate ordinary reflecting {uj,9)-ray and 
let UJ ^ 0. Then under the assumption (j2.3p we have 



-T^ e sing suppt s{t,9,uj) (2.4) 
and for t close to —T-y the scattering kernel has the form 

1 \(»-l)/2 



2^) (-l)"--^exp(i-/?^) (2.5) 

j("-i)/2(-i _,_ 2^^) + lo^er order singularities. 



dct dJ-y{uj){i'{qi),uj) -1/2 



(i^(g„0,6') 

_ffere is the number of reflections ofj, qi {resp. qm) is the first {resp. the last) 
reflection point of ^ and (3-y G Z. 

For strictly convex obstacles we have 

- 1 /? - ~ 1 _ 
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9 — uj is parallel to v{qi) and 

\det dJ^^{u^J\ =4\e -uj\''-^IC{x+), 

where 7+ is the unique (oj, 0)-reflecting ray at x+, u+ is the corresponding point on 
and IC{x+) is the Gauss curvature at x+. Thus we obtain the result of Majda 
[5] (see also describing the leading singularity at —T^^- 

To obtain an equality in the Poisson relation (|2.ip . one needs to know that every 
(u;,6')-ray produces a singularity. To achieve this, a natural way to proceed would 
be to ensure that the properties p.2p . (|2.3p hold. It is clear, that these properties 
depend on the global behavior of the (uj, 0)-rays in the exterior of the obstacle, 
and in this regard the existence of (w, 0)-rays with tangent or gliding segments 
leads to considerable difficulties. Moreover, different ordinary reflecting rays could 
produce singularities which mutually cancel. By using the properties of (w, 0)-rays 
established in [15], [1^, as well as the fact that for almost all directions (w, 9), the 
(a;,^)-rays are ordinary reflecting (see ^9\), the following was derived in [l9| : 

Theorem 2.2. (|19j) There exists a subset TZ of full Lebesgue measure in §"^^ x 
§"^^ such that for each {to, 9) £ TZ the only {uj,9)-rays in ft are ordinary reflecting 
{lu, 9)-rays and 

sing suppj s{t,9,uj) = {-T^ : 7 e Cuj,e{^)} ■ 
This result is the basis for several interesting inverse scattering results (see |20) . 

m)- 

3. Trapping obstacles 

Given a generalized bicharacteristic 7 in S*{Q), its projection 7 =^ (7) in E;, is 
called a compressed generalized bicharacteristic. Let Uo be an open ball containing 
K and let C be its boundary sphere. For an arbitrary point z ~ (x,^) £ Sb, 
consider the compressed generalized bicharacteristic 

7,(t) = (a;(t),e(t))eSb 

parametrized by the time t and passing through z for t = 0. Denote by T(z) G 
M+ U 00 the maximal T > such that x{t) e Uq for < t < T{z). The so called 
trapping set is defined by 

Soo = {{x,0 e Sb : a; e C, T{z) = 00} . 

It follows from the continuity of the compressed generalized Hamiltonian flow that 
the trapping set Sqo is closed in S;,. For simplicity, in the following the compressed 
generalized bicharacteristics will be called simply generalized ones. The obstacle 
K is called trapping if Eco 7^ 0, i-e. when there exists at least one point (a:,^) S 
C X §"^^ such that the (generalized) trajectory issued from (x, ^) stays in Uq for all 
< > 0. This provides some information about the behavior of the rays issued from 
the points (j/,77) sufficiently close to (x,^), however in general it does not yield any 
information about the geometry of (w, 0)-rays. 

Now for every trapping obstacle we have the following 

Theorem 3.1. ([15j. [18j ) Let the obstacle K be trapping and satisfy the condition 
(Q). Then there exists a sequence of ordinary reflecting {ijjm,9m)-rays jm with 
sojourn times T-y^ — > 00. 
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To prove this we use the foUowing 

Proposition 3.2. ([7], ^) The set of points {x,£,) € S^{n) = {{x,^) £ T*(f7) : 
X £ C, 1^1 ~ 1} such that the trajectory {J-t{x,S,) : ^ > 0} issued from (x,^) is 
bounded has Lebesgue measure zero in S^^fl). 

Proof. Assume K is trapping and satisfies the condition (Q). We will establish the 
existence of (w, 0m)-rays with sojourn times Tm — > oo for some u £ S"~^ suitably 
fixed. It is easy to see that Sf, \ Soo ^ 0- Since K is trapping, we have Soo 7^ 0, 
so the boundary cJEoo of Eqo in Sf, is not empty. Fix an arbitrary z £ dT,oo and 
take an arbitrary sequence Zm = (0, Xm, 1, ■Cm) £ Sf,, so that Zm ^ "^oo for every 
m £ N and z^ — > z. Consider the compressed generalized bicharacteristics Sm = 
(t, Xmit), l,Cm{i)) passing through Zm for i = with sojourn times T^^^ < 00. If the 
sequence {T^^} is bounded, one gets a contradiction with the fact that z £ Eoq. 
Thus, {Tz^} is unbounded, and replacing the sequence {zm} by an appropriate 
subsequence of its, we may assume that Tz,,, — > +00. Setting 

Vni = Xm{T{Zm)) £ C, W„i = ^m(r(z„)) £ §"~^ 

and passing again to a subsequence if necessary, we may assume that j/„j — > 
zq £ C, LUm ^ OJQ £ S"^^. Then for the generalized bicharacteristic = 
(t, a;(t), 1, ^(t)) issued from fj, — (0, zo,l,a;o) we have T{Sf^) = 00. Next, consider 
the hyperplane passing through z and orthogonal to ujq and the set of points 
u £ Zoo such that the generalized bicharacteristics 7„ issued from u £ Z^o with 
direction ujq satisfies the condition T{'-fu) = 00. The set Zoo H Z^^ is closed in 
and \ Zoo =/= 0- Repeating the above argument, we obtain rays 7m with so- 
journ times T^,^ — > +00. Using Proposition 13. 21 we may assume that each ray 7™ 
is unbounded in both directions, i.e. 7™ is an (ix'o,^m)-ray for some 0m £ 
Moreover, according to results in [1] and [16], these rays can be approximated by 
ordinary reflecting ones, so we may assume that each 7,„ is an ordinary reflecting 
{uJm, ^m)-ray for some ujm, £ ■ This completes the proof. □ 

To show that the rays 7„i constructed in Theorem 13.11 produce singularities, 
we need to check the condition (j2.3p . In general the ordinary reflecting ray 7^ 
could be degenerate and we have to replace 7^ by another ordinary reflecting non- 
degenerate ^'m)~^^y I'm with sojourn time sufhciently close to T^^. Our 
argument concerns the rays issued from a small neighborhood W C C x S"^^ of 
the point (zq, wq) G C x S"^^ introduced in the proof of Theorem 3.1. 

Let 0{W) be the set of all pairs of directions {ijj,e) £ §"-1 x §"-1 such that 
there exists an ordinary reflecting (w, 0)-ray issued from {x,uj) £ W with outgoing 
direction 6 £ S"~^. To obtain convenient approximations with (ct;,0)-rays issued 
from W , it is desirable to know that 0{W) has a positive measure in S"~^ x S"~^ 
for all sufficiently small neighborhoods W C C x §"~^ of {zq, ojq). Roughly speaking 
this means that the trapping generalized bicharacteristic <5^(t) introduced above is 
non-degenerate in some sense. More precisely, we introduce the following 

Definition 3.3. The generalized bicharacteristic 7 issued from (j/,77) £ C x §"~^ 
is called weakly non-degenerate if for every neighborhood W C C x S"^^ of {y,ri) 
the set 0{W) has a positive measure in §"~^ x §"~^. 

The above definition generalizes that of a non-degenerate ordinary reflecting 
ray 7 given in Section 2. Indeed, let 7 be an ordinary reflecting non-degenerate 
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(wq, 6'o)-ray issued from {xo,luo) G C x S" ^. Let Z — and consider the C°° 
map 

D = X xV ^ [x,uj) — > f{x,uj) e §"-\ 
where X d Z \s a. small neighborhood of a;o, F C §"^^ is a small neighborhood of 
Wqj and f{x,uj) is the outgoing direction of the ray issued from x in direction w. 
We have det/^(a;o,Wo) 7^ and we may assume that D is chosen small enough so 
that det/^(x,w) ^ for (x,cj) G -D. Set 

ma^_\mx,u;)r'\\^-. 

Then for small e > we have \\f^{x,uj) — /^(xo,wo)|| < f , provided ||a; — xo\\ < 
e, II w — cjoll < e and 

= {x G Z : llx - xoW <e}cX, T, ^ {uj e §""^ : \\uj - t^oll < e} C T. 

Next consider the set 

s, = {0e§"-i: ||^^-0o||<f }. 

Then taking e' G (0, e) so that l|/(xo,a;) — ^oH < ^ for w G T^i, and applying the 
inverse mapping theorem (see Section 5 in [H]), we conclude that for every fixed 
uj G r^/ and every fixed 9 G 2e' we can find a;(„ G X^^ with e), cj) = 6'. 

Consequently, the corresponding set of directions Fj/ x E^' C C'(W^) has positive 
measure in §"~^ x §"~^. This argument works for every neighborhood of {xo,uJo), 
so 7 is weakly non-degenerate according to Definition 13.31 



Remark 3.4. In general a weakly non-degenerate ordinary reflecting ray does not 
need to be non-degenerate. To see this, first notice that the set of those (y, rj) G 
CxS""^ that generate weakly non-degenerate bicharacteristics is closed in CxS"~^. 
Now consider the special case when K is convex with vanishing Gauss curvature 
at some point xo G dK and strictly positive Gauss curvature at any other point of 
dK. Consider a reflecting ray 7 in R" with a single reflection point at xq. Then, 
as is well-known, 7 is degenerate, that is the differential cross section vanishes. 
However, arbitrarily close to 7 we can choose an ordinary reflecting ray dm with 
a single reflection point Xm 7^ a^o- Then Sm is non-degenerate and hence it is 
weakly non-degenerate. Thus, 7 can be approximated arbitrarily well with weakly 
non-degenerate rays, and therefore 7 itself is weakly non-degenerate. 

Now we have a stronger version of Theorem 13.11 

Theorem 3.5. Let the obstacle K have at least one trapping weakly non-degenerate 
bicharacteristic 6 issued from {y, rf) G Cx§"~^ and let K satisfy (Q). Then there ex- 
ists a sequence of ordinary reflecting non- degenerate (wm, 9m)-rays 7™ with sojourn 
times T-y^ > 00. 

Proof. Let Wm C C x be a neighborhood of {y, rj) such that for every z G Wm 
the generalized bicharacteristic 7^ issued from z satisfies the condition 7(7^) > m. 
The continuity of the compressed generalized flow guarantees the existence of Wm 
for all TO G N. Moreover, we have Wm+i C Wm- Consider the open subset Fm of 
C X X C X S"~^ consisting of those {x,u),z,9) such that {x,lu) G Wm and 

there exists an ordinary reflecting (tj,6')-ray issued from {x,uj) G Wm and passing 
through z with direction 9. 
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The projection Fm 3 (a;, lo^ z, 9) — > {to, 9) is smooth and Sard's theorem imphes 
the existence of a set Dm C §"^^ x §"^^ with measure zero so that if [uj, 9) ^ Vm 
the corresponding (w, 0)-ray issued from {x,uj) € Wm is non-degenerate. Then the 
set 0{Wm) \ T^m has a positive measure and taking (10^,(^771) £ 0{Wm) \ T^m we 
obtain an ordinary reflecting non-degenerate (wm, ^m)-ray 5m with sojourn time 
Tm issued from Zm G W™. Next we choose 

q{m) > max{m + 1, Tm}, q{m) G N 

and repeat the same argument for Wq(^m) and -F'q(m)- This completes the proof. □ 

Remark 3.6. In general, a generalized trapping ray S can be weakly degenerate if 
its reflection points lie on flat regions of the boundary. In the case when K is a 
finite disjoint union of several convex domains sufficient conditions for a trapping 
ray to be weakly non-degenerate are given in [16j . On the other hand, we expect 
that the sojourn time of an ordinary reflecting ray 7 may produce a singularity 
of the scattering kernel if the condition (|2.3p is replaced by some weaker one. For 
this purpose one needs a generalization of Theorem 2.1 based on the asymptotics 
of oscillatory integrals with degenerate critical points. 

Now assume that 7 is an ordinary reflecting non-degenerate (a;,0)-ray with so- 
journ time T-y issued from (x,^) G C x S"~^. For a such ray the condition (j2.3|) is 
not necessarily fulfilled. Since 7 is non-degenerate, there are no (cj, 0)-rays 6 with 
sojourn time issued from points in a small neighborhood of (x,^). This is not 
sufficient for (|2.3p and we must take into account all (cu, 0)-rays. The result in [12] 
says that for almost all directions {to, 9) G S"~^ x S"^^ all (u;,6')-rays are reflecting 
ones and the result in [15] implies the property (|2.2p for the sojourn times of ordi- 
nary reflecting rays (w, 0)-ray, provided that {uj,9) is outside some set of measure 
zero. Thus we can approximate (w, 9) by directions (w', 9') for which the above 
two properties hold. Next, the fact that 7 is non-degenerate combined with the in- 
verse mapping theorem make possible to find an ordinary reflecting non-degenerate 
(u)' ,9')-ra,y 7' with sojourn time sufficiently close to so that (|2.2|) and ()2.3p 
hold for 7'. We refer to Section 5 in [TS] for details concerning the application of 
the inverse mapping theorem. Finally, we obtain the following 

Theorem 3.7. Under the assumptions of Theorem 3.6 there exists a sequence 
{ujm,9m) G S"^"'^ X S"^""^ and ordinary reflecting non-degenerate {u!m,9m)-fo,ys 7m 
with sojourn times Tm — > 00 so that 

- Tm e sing supp s(t,w„, 6*™), Vm e N. (3.1) 

The relation (13. 1|) was called property (S) in [15] and it was conjectured that 
every trapping obstacle has the property (S). The above result says that this is true 
if the generalized Hamiltonian flow is continuous and if there is at least one weakly 
non-degenerate trapping ray 6. The assumption that 6 is weakly non-degenerate 
has been omitted in Theorem 8 in [18] . 

4. Trapping rays and estimates of the scattering amplitude 

The scattering resonances are related to the behavior of the modified resolvent of 
the Laplacian. For ImA < consider the outgoing resolvent R{X) — (—A — A^)^^ 
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of the Laplacian in Q with Dirichlct boundary conditions on dK. The outgoing 
condition means that for / e C^{il) there exists g{x) G C^{M.") so that we have 

R{X)f{x) ^ Ro{X)g{x), |x| ^ oo, 

where 

i?o(A) = (-A - A2)"1 : L2,^p(M") 
is the outgoing resolvent of the free Laplacian in K" . The operator 

R{X) : iL„p(f^) 3 / ^ RWf e hU^) 
has a meromorphic continuation in C with poles Aj, ImAj > 0, called resonances 
([?]). Let X G C^(M") be a cut-off function such that ^(^c) = 1 on a neighborhood 
of K. It is easy to see that the modified resolvent 

has a meromorphic continuation in C and the poles of Rx{X) are independent of 
the choice of x- These poles coincide with their multiplicities with those of the 
resonances. On the other hand, the scattering amplitude a(A, 6, lo) also admits a 
meromorphic continuation in C and the poles of this continuation and their multi- 
plicities are the same as those of the resonances (see [7]). From the general results 
on propagation of singularities given in [llj . it follows that if K is non-trapping, 
there exist e > and d > so that Rx{X) has no poles in the domain 

[/,^ = {A £ C : < ImA < elog(l + \X\) - d}. 

Moreover, for non-trapping obstacles we have the estimate (see [M] ) 

l|i?x(A)l|L^(o)^L^(o) < VA e f/,,d. 

We conjecture that the existence of singularities i„i — > — oo of the scattering 
kernel s{t,9rmi^m) implies that for every e > and d > we have resonances in 

Here we prove a weaker result assuming an estimate of the scattering amplitude. 

Theorem 4.1. Suppose that there exist meN, a > 0, e>0, d > and C > so 
that a(A, 6, u) is analytic in Uc.d cind 

|a(A,6',w)| < C(l + |A|)"e"|i'"^l, V(t^, 6*) £ S""! x S"-\ VA £ t/,,^. (4.1) 

Then if K satisfies (G), there are no trapping weakly non- degenerate rays in fl. 

The proof of this result follows directly from the statement in Theorem 2.3 in 
[15j . In fact, if there exists a weakly non-degenerate trapping ray, we can apply 
Theorem 3.7, and for the sequence of sojourn times {—Tm}, T„i oo, related to 
a weakly non-degenerate ray S, an application of Theorem 2.1 yields a sequence 
of delta type isolated singularities of the scattering kernel. The existence of these 
singularities combined with the estimate (|4.ip leads to a contradiction since we may 
apply the following 

Lemma 4.2. (^15j) Let u £ 5'(M) be a distribution. Assume that the Fourier 
transform m(OjC ^ admits an analytic continuation in 

W,A = £ C : rf-elog(l + 1^1) < Im^ < 0}, e > 0, d > 

such that for all ^ £ W^^d we have 

KOI<c(i + |^|A^|i'"«i,7>o. 
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Then for each q G N there exists tq < t and Vq G C^(R) such that u ~ Vq for t <tq. 

Here the Fourier transform u{£) / e-'^^u{t)dt for u e C^(M) and for A G M 
we have 

/iA\(«-i)/2— /iA\("-i)/2 , 

s(A,0,c.) = (— J a(A,0,c.)- (— ) ai-X,9,Lu). 

Thus s(A,6',w) admits an analytic continuation in We.d and the estimate (I4.ip im- 
phes an estimate for s(A,6',u;) in W^^d- 

It is easy to see that the analyticity of in C^e.d and the estimate 

l|i?x(A)||L^(o)^L^(o) < + lAD^'e^'l^-^^l, VA G C/^ (4.2) 

with m' G N, a' > 0, imply (|4.ip with suitable to and a. This follows from the 
representation of the scattering amplitude involving the cut-off resolvent i?^(A) 
(see [H], [n]) with ^ e C(f (M") having support in {x G R" : < a' < < b'}. 
Moreover, we can take a' < h' arbitrary large. More precisely, let G C^(R") be 
a cut-off function such that </?a(2^) = 1 for \x\ < p. Set 

Fa (A, uj) = [A(^, + 2iA(Vv?a, c^)]e'^<"''"> . 

Let (/7b (x) G C^(]R") be such that (pb{x) = 1 on a neighborhood of K and (pa{x) — 1 
on supp tpb. The scattering amplitude a{X,0,Lo) has the representation 

a(A,0,^) = c„A("-3)/2 f e-'^<--'^\{Aipb)RiX)Fa{\,ij) 

+2(V,v?b, V.(i?(A)Fa(A, cj)))] da: 

with a constant c„ depending on n and this representation is independent of the 
choice of and In particular, if the estimate (|4.2p holds, then the obstacle K 
has no trapping weakly non-degenerate rays. 

Consider the cut-off resolvent R^{X) with supp ■0 C {a; G M" : < a' < |a;| < 6'}. 
For A G M and sufficiently large a' and h' N. Burq [2] (see also [4]) established the 
estimate 

l|i?^(A)||L^(o)^L^(o) < A G M (4.3) 

without any geometrical restriction of K. On the other hand, if we have reso- 
nances converging sufficiently fast to the real axis, the norm \\Rx{^)\\L'^(n)~>L'^(n) 
with X = 1 on if increases like 0{e^^^^) for A G K, |A| — > oo. Thus the existence of 
trapping rays influences the estimates of RxW with xi^) equal to 1 on a neighbor- 
hood of the obstacle and the behaviors of the scattering amplitude a{\,9,uj) and 
the cut-off resolvent RxW for A G M are rather different if we have trapping rays. 

It is interesting to examine the link between the estimates for a{X,9,uj) and the 
cut-off resolvent R^iX) for A G U^ d- In this direction we have the following 

Theorem 4.3. Under the assumptions of Theorem 4.1 for a{X,9,uj) the cut-off 
resolvent R^iX) with arbitrary x G C^(R") satisfies the estimate ^4-^ t^e.d with 
suitable C" > 0, m' G N and a' > 0. 
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Proof. The poles of a{X,9,uj) in {z G C : ImA > 0} coincide with the poles of the 
scattering operator 

SiX) = / + K{X) : L2(§«-1) ^ i2(g«-i)^ 

where K{X) has kernel a{X,d,u!). Thus the estimate (I4.ip of a(A, leads to an 
estimate of the same type for the norm of the scattering operator S{X) for A S Ue^d- 
Notice that S~^(X) = S*{X) for every A G C for which the operator S'(A) is invert- 
ible. Moreover, the resonances Aj are symmetric with respect to the imaginary axe 
iM. 

Consider the energy space H ~ H£,{n) L^(f7), the unitary group U{t) — e'"^ 
in H related to the Dirichlet problem for the wave equation in Q and the semigroup 
Z''{t) = PlU{t)P^_, t > 0, introduced by Lax and Phillips ([7J). Here P| are the 
orthogonal projections on the orthogonal complements of the Lax-Phillips spaces 
-Dj-, b > p (see [7 for the notation). Let B'' be the generator of Z''{t). The 
eigenvalues Zj of are independent of b, the poles of the scattering operator 
S{X) are {—izj G C, Zj £ spec B''} and the multiplicities of zj and —izj coincide. 
Given a fixed function x e C^(R"), equal to 1 on K, we can choose 6 > so that 
P±X = xP± = X- We fix 6 > with this property and will write below B, P± instead 
of B'', P^. Changing the outgoing representation of if, we may introduce another 
scattering operator iS'i(A) (see Chapter III in [7]) which is an operator-valued inner 
function in {A G C : Im A < 0} and 

l|5'i(A)|U2(s,.-i)^i2(s„-i) < 1, ImA < 0. (4.4) 

The estimates (|4.4[) is not true for the scattering operator S{X) = I + K(X) related 
to the scattering amplitude. On the other hand, the link between the outgoing 
representations of H introduced in Chapters III and V in [7] implies the equality 

Si{X)^e-''^^S{X), I3>0. (4.5) 

The following estimate established in Theorem 3.2 in |7] plays a crucial role 

\\{iX-B)-'\\H^H < ^^^ll^r'(A)||L^(S"-i)-.L^(S"-i), VAGf/,,,\K- 

Since S^^{X) — S*{X) for all A G C for which S{X) is invertible, the estimates 
(grD and (g^D imply 

3g/3|ImA| 

ll(iA-B)" Wh^h < -^j^^^^ll'5'(A)||L2(s„-i)_i2(s„-i) 

^ „a'\ Im A| 

< Ci(l + |A|)™ VA G U,,d \ K. 

For Re A > we have 

nOO 

x{X~B)-\^ / e-^*x^+C/(i)P-xdt = -ix(-iA-G)-ix 

and by an analytic continuation we obtain this equality for A G 'iUe,d- By using the 
relation between -R^(A) and x(A — G)~^X: we deduce the estimate 



PxWllL^(n)^L^(n) <C2(1 + |A|)^ 



gtt' I Im A| 



|ImA| 
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for Im A = elog(l + |A|) — d, |RcA| > cq. On the other hand, Iji?;,- (A) 11^2(^)^^2(0) 
is bounded for ImA = — ci < and have the estimate (see for example [22]) 

\\R^iX)\\mn)^LHn) < Ce^'^'", ImA < elog(l + |A) - d. 
Then an apphcation of the Pragmcn-Lindclof theorem yields the result. 

□ 

It is an interesting open problem to show that the analyticity of a{X,9,uj) in 
Ue^d implies the estimate (|4.ip with suitable m, a and C without any informa- 
tion for the geometry of the obstacle. The same problem arises for the strip 
V5 = {AgC: 0< ImA < 6} and we have the following 

Conjecture. Assume that the scattering amplitude a(A, is analytic in Vs. 
Then there exists a constants Ci > 0, C > such that 

\a(X,e,uj)\ < Cie^l^l', y{uj,9) e §"-i x §"-\ VA G Vs. 

For n = 3 this conjecture is true since we may obtain an exponential estimate 
©(e*^!^!') for the cut-off resolvent i?x(A), \ &Vs (see for more details [1]). 
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